Optimal weighted averaging pre-processing schemes for laser absorption spectroscopy

ABSTRACT

A method of processing raw measurement data from a tunable diode laser absorption spectroscopy (TDLAS) tool or other spectroscopic instrument is provided that determines what types of noise (electronic or process flow) are present in the measurement. Based on that determination, the noise is reduced by performing a weighted averaging using weights selected according to the dominant type of noise present, or a general case is applied to determine weights where neither noise type is dominant. The method also involves performing continuous spectroscopy measurements with the tool, with the data and weighted averaging being constantly updated. Weighting coefficients may also be adjusted based on similarity or difference between time-adjacent traces.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/005,384, filed Jan. 25, 2016, the entiredisclosure of which is incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to spectroscopic measurement and moreparticularly to pre-processing of raw data from such measurement toreduce noise and remove unwanted artifacts from the raw data.

BACKGROUND ART

One of the problems occurring in spectrometry is noise that arises fromelectronic sensor fluctuations, illumination fluctuations, as well astime-varying transmission due to process flow variations, densitygradients, dust, and the like. Conventional tunable diode laserabsorption spectroscopy (TDLAS) systems and many other spectroscopicmeasurement systems often experience degradation in performance due tovarious noise sources.

A typical prior art spectrometry measurement is described, for example,in U.S. Pat. No. 8,711,357 to Liu et al, which provides for a referenceand test harmonic absorption curves from a laser absorption spectrometerhaving a tunable or scannable laser light source and a detector. Theabsorption curves are generated by the detector in response to lightpassing from the laser light source through respective reference andsample gas mixtures. The reference curve might be determined for thespectrometer in a known or calibrated state. The shape of the testharmonic absorption curve is compared with that of the referenceharmonic absorption curve and parameters of the laser absorptionspectrometer are adjusted to reduce the difference and correct the testcurve shape.

U.S. Pat. No. 6,940,599 to Hovde describes, spectral demodulation inabsorption spectrometry, wherein the spectral response of a detector canbe characterized in Fourier space.

However, neither of these methods deals directly with the presence ofnoise in the absorption signal. They would each perform better if thenoise could first be reduced or removed from the signal obtained fromthe detector.

Typically, conventional spectroscopic signal analysis will pre-processthe signal using simple averaging of successive measurements or tracesobtained\from multiple scans of the laser wavelengths. It does not useany weighted average or determine any optimal weighting scheme based onrealistic noise models. As such, while this simple averaging may beadequate in laboratory or ideally-controlled conditions, in more harshor uncontrolled measurement environments it may suffer unacceptabledegradation of performance due to an inability to reduce the noise inthe signal sufficiently. Beyond a certain limit, the systems can nolonger operate and report data reliably due to insufficient noisesuppression. This is a recurring problem for any field-deployed TDLASsystems or any spectroscopic system used in an environment where harshreal-world conditions such as dust, density gradients, temperaturegradients, and mechanical movement can lead to noise.

It is desired, to provide processing schemes that can suppress noise soas to improve overall sensor performance and extend the range ofconditions under—which spectroscopic systems can reliably operate. Noiselevels from a variety of sources due to electronics, laser fluctuation,or time-varying transmission through the process flow should be able tobe successfully reduced.

SUMMARY

A signal processing scheme uses a realistic noise model to assess atunable diode laser absorption spectroscopy (TDLAS) or otherspectroscopic measurement signal when that signal undergoes amplitudevariations from the various noise sources from (a) laser intensityfluctuations, (b) process flow variations (e.g., dust, density gradientsand other time-varying transmission losses), and (c) electronic anddetection system noise. An optimal averaging strategy depends upon thetype of noise expected and found by the analyzer system. The noise modelaccounts for the presence of any or all of these potential noise sourcesto select a weighted averaging scheme that is optimized according towhich, if any, of the noise sources happens to be dominant under theprevailing measurement conditions.

The method of processing spectroscopic raw data, such as tunable diodelaser absorption spectroscopy (TDLAS) measurement signals, begins byobtaining the raw data, e.g. by measuring the amplitude of each laserscan. Then, optimal weighting coefficients are chosen based on whetherthe noise is dominated by electronic sources independent of the measuredvolume, or the noise is dominated by changes in the measurement volume,or a general case where neither noise source dominates the other. Ifdetector noise is negligible and process flow fluctuations dominate, thereceived signal should be normalized by amplitude. On the other hand, ifdetector noise dominates, the received signal should be—multiplied bythe amplitude. If both effects contribute significantly, a genericformula can set the appropriate weights. After completing a weightedaverage calculation using the determined optimal weighting coefficients,the resulting spectrum is analyzed to produce the desired measuredquantity.

The measurement noise to be reduced by the appropriate choice ofweighting coefficients may be caused by any one or more of electricalsources, process flow variations, attenuation due to dust orparticulates, density gradients in the flow, or other time-varyingtransmission losses. Measurement of the amplitude of each laser scan maybe determined by any of peak-to-peak signal amplitude, signal amplitudeat a specific measurement point or during a specific duration of timeduring the TDLAS trace, signal amplitude with a background signalsubtracted, or slope of the ramp portion of the TDLAS trace.

Weighting factors may also be adjusted based upon similarity totime-adjacent traces either prior or post to the current measurementtrace. To make such adjustments, the system can calculate thedifferences between the current trace and the prior (and/or post) trace,squaring those differences between the two traces, calculating anadditional weighting factor by summing and taking the inverse of thesum, then additional weighting factor by the original weighting factor.

Advantages of this pre-processing scheme include (a) improved sensorperformance; (b) reduction in noise of measured values reported by thespectroscopy instrument; and (c) resistance to laser power fluctuations,dust, beam steering, density gradients, temperature gradients, andmechanical movement in the process that would otherwise lead todegradation of sensor performance. The weighted average, calculatedbased on the noise model parameters and parameters of the measuredsignal, reduces noise below that which is achievable by the traditionalequal-weighted simple averaging scheme. In this way, performance (i.e.precision and accuracy) of TDLAS and other spectroscopic instruments isenhanced so that such instruments can be deployed even in harshenvironments so as to successfully measure data near its limits ofdetectability.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a graph of sensed light amplitude versus time representingraw data from a TDLAS instrument collected over ten successivewavelength scans.

FIG. 2 shows a graph of optical power generated by a TDLAS laser,normalized by its maximum power, versus time for a single transmittedramp.

FIG. 3 is a block diagram of a simulation of the overall procedure,which represents the basic approach applicable to TDLAS or otherspectroscopic instruments employing the present invention.

FIG. 4 is a graph of performance of different averaging approaches(simple average, multiplied average, normalized average, weights inversedistance) in terms of standard deviation of concentration (ppm) versusmodulation frequency (kHz).

FIG. 5 illustrates an embodiment of a tunable diode laser absorptionspectroscopy (TDLAS) system.

FIG. 6 illustrates an embodiment of a process of measuring using a TDLASsystem, the process being adaptable to incorporation of the weightedaveraging scheme of the present invention in step 270.

FIG. 7 illustrates an example of a process of pre-processing the rawdata from a TDLAS system in accord with the present invention.

FIG. 8 illustrates an embodiment of a process of operating a TDLASsystem using the data pre-processing of FIG. 7.

DETAILED DESCRIPTION

A data averaging scheme is presented to optimally process rawspectroscopy data, such as a tunable diode laser absorption spectroscopy(TDLAS) signal when that signal undergoes amplitude variations caused byprocess flow variations (e.g. dust, density gradients, and othertime-varying transmission losses) or by measurement noise. A model isdeveloped which accounts for noise sources from (a) laser intensityfluctuations, (b) process flow variations, and (c) electronic anddetection system noise. The model is used to derive an optimalsignal-averaging scheme for both noise sources and the generic casecombining both noise sources. A weighted average is calculated based onthe model parameters and parameters of the measured shot-by-shot TDLASsignal and is used to reduce the measurement noise below that which isachieved using traditional equal-weighted averaging. In this way, theperformance (e.g. precision and accuracy) of the TDLAS or otherspectroscopic instrument can be enhanced in harsh environments whereprocess flows induce large transmission changes and when the instrumentis used to measure data near its limits of detectability (LOD).

In spectroscopic measurement, for example with TDLAS, the raw datarepresenting the light received by the photo-detector can contain strongnoise and artifacts due to fast transmission fluctuations. Clearly,these data should not be fed directly to the curve fitting algorithm forquantitative analysis of the gas concentration, but a carefully selectedpre-processing is necessary to reduce the noise and remove unwantedartifacts.

An instance of the raw data collected from a TDLAS instrument is shownin FIG. 1. Clearly, one frequency scan 10 in FIG. 1 lasts 1 ms. On theother hand, the gas analysis must be performed, at most every 100 ms-ls.Therefore, a possible way for increasing the signal to noise ratio andremoving artifact is by performing a form of averaging of the raw data.Let us denote with {x_(i,j)} for i=1, . . . , N and j=1, . . . M theTDLAS raw data. In particular x_(i,j) represents the i^(th) samples ofthe light received by the photodetector during the j^(th) scan. Nrepresents the number of samples per scan, while M the number of scans.Given the raw data {x_(i,j)}, the goal is to design the weights{w_(i,j)} such that the averaged data, denoted as x _(i) defined asfollowsx _(i)=Σ_(j=1) ^(M) w _(i,j) x _(i,j)  (1)have maximal signal-to-noise ratio and contain no artifacts. After thepre-processing step, the averaged signal is ready to be fed into thesignal processing scheme in which the curve fitting and theconcentration determination are performed.

On top of the electrical noise (due to both TDLAS transmitter andreceiver), it has been shown that dust present in the duct will causetransmission fluctuations. As a consequence, the ramps may not only havea different amplitude but some of the ramps may even be stronglydistorted by fast transmission fluctuations. It is not obvious whetherramps with low amplitudes should be simply added or down-weighted withrespect to ramps having large amplitude. Moreover, it is not immediatelyclear whether, it is appropriate to discard or down-weight heavilydistorted ramps. In the ensuing sections, these alternatives areexplored and the performances obtained using several pre-processingschemes are assessed.

Weighted, Average Based on Signal Amplitude

The electrical ramp signal 20 delivered to the laser driver is shaped tohave two constant plateaus 21 and 22 which can be used to evaluate theramp offset and amplitude. An instance of the transmitted ramp isdepicted in FIG. 2. (In principle, the ramp structure could be changed,and in fact, the second plateau 22 could be avoided in someembodiments.)

As shown in FIG. 1, after light transmission and detection, differentscans might have different offsets and different amplitudes. Given theraw data, the offsets of each scan can be calculated via the firstplateau since the laser, during this time, will not transmit light. Theoffset of the j^(th) scan is calculated as follows:

$o_{j} = {\frac{1}{i_{1,{stop}} - i_{1,{start}} + 1}{\sum\limits_{{i = {i\;}_{1}},{start}}^{i_{1,{stop}}}x_{i,j}}}$Where i_(1,start) and i_(1,stop) are the sample index where the firstplateau start and stop, respectively. Successively, the offset can beremoved as follows:x′ _(i,j) =x _(i,j) −o _(j)After having calculated and removed the offset, the amplitude of eachramp can be estimated by calculating the average amplitude of the secondplateau. Denoting A_(j) the amplitude that is experienced by the scan j,A_(j) can be calculated as follows:

$A_{j} = {\frac{1}{i_{2,{stop}} - i_{2,{start}} + 1}{\sum\limits_{{i = {i\;}_{2}},{start}}^{i_{2,{stop}}}x_{i,j}^{\prime}}}$where i_(2,start) and i_(2,stop) are the sample index where the secondplateau start and stop, respectively. From now on, the amplitude of eachramp can be considered known.

Assuming further that this amplitude keeps constant over the duration ofthe ramp, the effect of simple average, multiplied average andnormalized average on the signal variance is assessed. In synthesis, itholds that the pursuit of the optimal weighting scheme depends on thesource of noise. In particular, if we assume that the major noisecontribution is additive to the acquired ramp (due to additiveelectronic noise at the detector or due to fluctuations), the optimalaveraging scheme is to weigh the ramp by the respective signalamplitude, i.e., ramps with large amplitude will contribute more to theaverage than ramps with small amplitude, which are down-weighted. Theanalysis can also predict the expected loss in variance if the simpleaverage scheme is used instead of the weighted average scheme. It isworth pointing out here that this loss can be very minimal especially ifthe signal amplitude does not significantly vary among different ramps.

In synthesis, we can identify three schemes of weighting based on signalamplitude (It is worth pointing out here that the weights do notnecessarily have to sum to 1, since, eventual multiplication factors canbe compensated by the successive block of baseline fitting):

W1) Simple Average

The weights in the eq. (1) are defined as follows:w _(i,j)=1This represents the conventional approach of the prior art.W2) Multiplied AverageThe weights in the eq. (1) are defined as follows:W _(i,j) =Aj for each i=i _(r,start , . . . ,) Nwhere i_(r,start) is the index wherein the linear ramp starts. Themultiplied average weights are optimal when the noise is additive, i.e.,noise due to fast fluctuation due to dust or electronic noise.W3) Normalized AverageThe weights in the eq. (1) are defined as follows:

$W_{i,j} = {{\frac{1}{Aj}\mspace{14mu}{for}\mspace{14mu}{each}\mspace{14mu} i} = {i_{r,{start},\ldots,}N}}$The normalized average weight is optimal in case the additive noise canbe neglected and the noise is dominated by change in absorbance, i.e.,pressure, effective path length, temperature, and concentration.Weighted Average Based on Signal Similarity

As anticipated, fast and strong fluctuations might heavily distort asingle ramp signal. The question to answer is whether todiscard/down-weight distorted ramps or not. Several weighting schemesfor down-weighting distorted ramps can be envisioned. In particular, aheuristic approach that has been found effective, is to calculate thedistance among consecutive ramps (discarding the points of the twoplateaus). The distance is calculated in the L2 norm sense, i.e. thepoints of two consecutive ramps are subtracted, the difference vector isthan squared, and the squared differences are summed up. More precisely,given a certain ramp, the distances with respect to the previous andsubsequent ramps are calculated, and summed up. The weight is then theinverse of this quantity. In fact, if the distances of the current rampwith respect to the previous and subsequent ramps are small, the weightto the ramp in the average will be large. On the other hand, if the rampis very distant from the adjacent ramps, then the weight will be smalland the ramp contribution in the average is small.

In synthesis, we can identify two schemes of weighting based on signalsimilarity:

W4) Weights Inverse Distance

The weights in the eq. (1) are defined as follows:

$w_{i,j} = \frac{1}{{\sum\limits_{{i = i_{r}},{start}}^{N}\left( {x_{i,j}^{\prime}-=x_{i,{j - 2}}^{\prime}} \right)^{2}} + {\sum\limits_{{i = i_{r}},{start}}^{N}\left( {x_{i,j}^{\prime} - x_{i,{j - 2}}^{\prime}} \right)^{2}}}$for each i=i_(r,start, . . . ,) N. Clearly Σ _(i=i) _(r) _(,start)^(N)(x′_(i,j)−x′_(i,j−1))² is the square Euclidean distance between thej^(th) and (j−1)^(th) ramps, while Σ_(i=i,start)^(N)(x′_(i,j)−x′_(i,j+1))² is the square Euclidean distance between thej^(th) and (j+1)^(th) ramps. Numerical tests have shown that the Weightsinverse distance selection improves the performance in presence of fastfluctuation due to, e.g., dust.W5) Weights Inverse Local DistanceThe weights in the eq. (1) are defined as follows:

$v_{i,j} = \frac{1}{\left( {x_{i,j}^{\prime} - x_{i,{j - 2}}^{\prime}} \right)^{2} + \left( {x_{i,j}^{\prime} - x_{i,{j + 2}}^{\prime}} \right)^{2}}$for  each  i, j  and$w_{i,j} = \frac{v_{i,j}}{\sum\limits_{j - 1}^{M}v_{i,j}}$The main difference of W4 and W5 is that, if one ramp contains a largeoutlier, in the W4 case, the whole ramp is down-weighted, while in theW5 case, only the sample containing the outlier and the adjacent pointsare down-weighted.Remark: In principle, the averaging rules W4 and W5 could be applied tothe original signal x_(i,j) instead of x′_(i,j). In addition, the samerules could be applied to the standardized raw data (the raw data afteroffset removal and normalization), that is,

$x_{i,j}^{''} = \frac{x_{i,j}^{\prime}}{A_{j}}$Finally, the weighted average based on signal similarity can also becombined with the weighted average based on signal amplitude. Here, thetotal weight can be evaluated by, e.g. multiplication of the weightsbased on signal amplitude and on signal similarity.For the facilitating IP protection, we could also include a generalcase, defined as follows:W6) General Weights

$w_{i,j} = {f\left( \left\{ x_{i,j} \right\}_{\underset{{j = 1},\;{\ldots\; N}}{{i = 1},\;{\ldots\; N}}} \right)}$Simulation ResultsPre-Processing Results Obtained with Weighted Average Based on SignalAmplitude

We have performed a quantitative analysis of the resulting concentrationvariance versus modulation frequency and averaging scheme. Theperformance is measured in terms of concentration variance. Oneimportant implementation trick was the utilization of a warm start,i.e., the previous (averaged) ramp parameters calculated by the curvefitting are fed as starting point of the new curve fitting for the nextaveraged ramp. For this procedure to be effective, the ramp isnormalized by the second plateau level before being fed to the curvefitting algorithm.

The block scheme of our simulation is depicted in FIG. 3. FIG. 4 showsthe concentration variance versus modulation frequency for the simpleW1, normalized W3, and weighted averaging schemes W2 and W4. Increasingthe modulation frequency, as expected, has a positive influence on theconcentration variance since it significantly decreases. As expected,the multiplied average W2 is the most promising strategy, but theimprovement with respect to the simple average W1 is only minimal, andthe performance of all averaging schemes coincides for high modulationfrequencies (i.e., 8 kHz).

In conclusion, with the current dust transmission data, the gain of themultiplied average scheme with respect to simple averaging is onlyminimal (i.e., 5%), while the two schemes perform almost identically at8 kHz. These results indicate the effectiveness of performing additionalweighted averaging based on signal similarity.

Spectroscopic Instrument and Measurement Process

With reference to FIG. 5, a spectroscopy system 100 capable of employingthe signal pre-processing scheme according to the present invention mayinclude a laser control module 110, laser diode(s) 120, sample chamber130 and (optional) reference chamber 135, corresponding detectors 140and detection electronics 150, a digital signal processing module 160and a computer 170. System 100 thereby provides for spectroscopymeasurements of samples in comparison with control substances and forrecording and display of results. Of particular interest are the digitalsignal processing module 160 and computer 170 that process spectroscopicdata received from the detection electronics 150 in the manner describedbelow.

Laser control module 110 controls one or more laser diodes 120. Laserdiodes 120 are tunable laser diodes, providing for a range of outputs.Sample chamber 130 receives a sample, typically in a gaseous or vaporform, for evaluation. If provided in the system, a reference chamber 135similarly receives a reference substance or reference sample forevaluation in comparison to the sample of sample chamber 130. Laserdiodes 120 are arranged to illuminate sample chamber 130 and referencechamber 135, such as through an arrangement of mirrors, lenses andwindows (not shown), repeatedly sweeping through a specified range ofwavelengths over a period of time. Detectors 140 are positioned toreceive light from laser diodes 120 after the light passes through thesample chamber 130 (and optional reference chamber 135) and after thelight has undergone wavelength-specific absorption by atomic ormolecular species present in the chamber(s). Thus, detectors 140 detectpresence of specific atoms or molecules in reference chamber 135 andsample chamber 130 responsive to light from laser diodes 120.Arrangement and construction of laser diodes 120, chamber 130 and 135and detectors 140 is well understood to those having skill in the art.

Detection electronics 150 operate detectors 140 and record data receivedfrom detectors 140 in raw format. Digital signal processing module 160processes raw sample data from the detection electronics 150 in themanner described below to extract usable measurement data while removingor substantially reducing various noise contributions present in the rawdata. Computer 170 controls operation of the digital signal processingmodule 160 and laser control module 110, as well as possibly other partsof the system 100, such as the mirrors, lenses and windows mentionedabove, and flow of sample materials into chambers 130 and 135, andthereby controls operation of the overall system 100.

Division of processing between detection electronics 150, digital signalprocessing module 160 and computer 170 may vary depending on exactimplementations in various embodiments. However, one may expect thesethree modules to collectively perform processes related to recordingdata, processing raw sample data into measurement data. Moreover,computer 170 may be expected to store data points in various formats(raw sample data and processed measurement data) for longer-term storageand display purposes. Digital signal processing module 160 may include asignal digital signal processor (DSP) such as those available from TexasInstruments for example. Digital signal processing module 160 mayinclude more elaborate components such as multiple DSPs and relatedcomponents.

A process 200 like that in FIG. 6 may be implemented by anyspectroscopic system, such as the system 100 of FIG. 5, or a similarsystem using tunable laser diodes or other tunable lasers, or even abroad spectrum instrument with wavelength separation onto multipledetectors. The result, however obtained, is raw sample data representingwavelength-specific optical absorption over some wavelength range, wheremeasurement noise may be present from any of additive electronicdetector noise, laser intensity fluctuations, density gradients or dustin the sample flow, or small fluctuations in the underlying absorptiondue to pressure and temperature variations of the sample. The raw sampledata is processed to remove or reduce as much of the measurement noiseas possible in order to obtain usable measurement data.

Process 200 may generally include initiating operation 210 of aspectroscopy instrument, providing a test sample (220) and perhaps alsoa control sample (230), illuminating laser diodes 240, which may bedriven to produce a series of sweeps over a specified range ofwavelengths, detecting sample data points (250, 260), pre-processing thesample data 270 according the method of the present invention to removenoise contributions, and recording and presenting processed data points280. Various steps of the process 200 may be executed or implemented ina variety of ways, whether by a pre-programmed machine, a specializedmachine, or a set of machines.

Initiating the operation in step 210 may include any warm-up andcalibration procedures that may be necessary, along with possiblepreparation and placement of samples (depending on their source). Forexample, some components may need to be set to a controlled temperaturefor preferred operation, and chambers may need to be pumped down tonear-vacuum. Step 220 (and optional step 230) may involve the flowing ofa gaseous (or vaporized) test sample into the spectroscopy instrument'stest chamber, and likewise flowing gaseous (or vaporized) control sampleinto any parallel control chamber that the instrument may have.

In step 240, measurement can begin, as the laser diode(s) areilluminated. This may involve simply illuminating a single tunable laserdiode, or may turn on different laser diodes in a series fashion fordifferent wavelength bands, or may involve illuminating multiple laserdiodes simultaneously with subsequent wavelength-separation of the lightwithin the spectrometer toward different detectors. Responsive to step240, raw data points are detected from the sample chamber (and thereference chamber) in step 250 (and optional step 260), based onabsorption of the laser light as it passes through the sample gas in therespective chambers. These raw data points are recorded at the time ofdetection.

At step 270, the raw data points are processed through digital signalprocessing to provide measurement data, which may be normalized, forexample, or otherwise' adjusted for any known systematic effects uponthe measurement. This also includes a weighted averaging in a manneraccording to the present invention that minimizes various possible noisecontributions to the data. In step 280, the processed measurement datais stored, analyzed, displayed and the like, as required by a user.

While conventional use of a TDLAS system or other spectroscopicinstrument is well understood with regard to FIG. 6, opportunities forbetter noise modeling and calibration exist, in particular in relationto step 270. To understand an approach implemented in variousembodiments herein, it may be useful to start with mathematicalbackground for a potential calibration process.

Let I₀(ν) and I_(k)(ν) denote the light emitted by the laser and thelight detected at the photodetector at wavenumber ν during the kth scan,respectively, for k=1, . . . , K where K represents the total number ofscans. According to the Beer-Lambert law, it holds thatI _(k)(ν)=I ₀(ν)e ^(−A(ν))  (2)where A(ν) is the absorbance at wavenumber v, and it is assumed to beconstant with the scan index. Since A(ν)≈0, a first-order Taylorexpansion of the exponential function in (2) yieldsI _(k)(ν)=I ₀(ν)[1−A(ν)]  (3)Unfortunately, (3) is only an abstraction and in practical cases severaldisturbances impair it. In fact,

-   1. additive detector noise is typically present in I_(k) (ν). Let us    denote such noise as e_(k)(ν);-   2. small fluctuations of the true underlying absorption can occur    due to, e.g., pressure and temperature variations. Let us denote    such noise as ν_(k)(ν);-   3. variation in the signal amplitude can occur due to either or both    laser intensity fluctuations and physical perturbations, e.g., dust    or density gradients in the flow. Let us assume that this amplitude    fluctuation is independent of n and let us denote such amplitude as    c_(k).    A possible model that includes the three disturbances mentioned    above is given as follows:    I _(k)(ν)=C _(k) I ₀(ν)[1−A(ν)+ν_(k)(ν)]+e _(k)(ν);  (4)    Denoting T(ν):=[1−A(ν)]I₀(ν), the model in (4) can be rewritten as    I _(k)(ν)=C _(k)[T(ν)+I ₀(ν)ν_(k)(ν)]+e _(k)(ν);  (5)

One can expect that the gain C_(k) is known for each k. This knowledgecan be achieved, e.g., by enabling the laser to send a constant plateauwhere the current is kept constant and by averaging the raw detectedlight during this constant interval. It is important to guarantee thatthere is no absorption at the wavenumber of the plateau.

One can focus on a specific wavenumber, ν*, and drop the wavenumberindex for notation simplicity. The averaging goal is to find, theoptimal weights {w_(k)}_(k−1) ^(k) such that the estimator Ī:=Σ_(k=1)^(k)w_(k)I_(k) is unbiased and has minimum variance. To carry out theanalysis, one can assume that both e_(k) and ν_(k) are normallydistributed around 0, i.e., e_(k)˜N(0, σ₀ ²), ν_(k)˜N(0, σ_(f) ²), andthat both noises are independent. Let us also define σ_(ν) ²:=I_(o)²σ_(f) ².

Optimal Averaging Scheme

Using (5), it holds that

$\begin{matrix}{{\sum\limits_{k - 1}^{k}{w_{k}I_{k}}} = {{\left( {\sum\limits_{k - 1}^{k}{w_{k}c_{k}}} \right)T} + \left( {\sum\limits_{k - 1}^{k}{w_{k}c_{k}I_{o}v_{k}}} \right) + \left( {\sum\limits_{k - 1}^{k}{w_{k}c_{k}}} \right)}} & (6)\end{matrix}$since e_(k)˜N(0, σ₀ ²) and, ν_(k)˜N(0, σ_(f) ²), it holds that

$\begin{matrix}{T\text{∼}{N\left( {{\left( {\sum\limits_{k - 1}^{k}{w_{k}c_{k}}} \right)T},{\sum\limits_{k - 1}^{k}{w_{k}^{2}\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{v}^{2}} \right)}}} \right)}} & (7)\end{matrix}$Therefore, the problem of finding the minimum unbiased estimator can becast as the following optimization problem:

$\begin{matrix}{{{\{\}}_{k - 1}^{k}:={\underset{{\{ w_{k}\}}_{k - 1}^{k}}{\arg\;\min}{\sum\limits_{k = 1}^{k}{w_{k}^{2}\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{v}^{2}} \right)}}}}{{{subject}\mspace{14mu}{to}\mspace{14mu}{\sum\limits_{k = 1}^{k}{w_{k}c_{k}}}} = 1}} & (8)\end{matrix}$To solve equation (8), we can resort to Lagrangian optimization. Writingthe Lagrangian of the problem in (8) yields

$\begin{matrix}{{L\left( {\left\{ w_{k} \right\}_{{k = 1},}^{k}\lambda} \right)}:={{\sum\limits_{k - 1}^{k}{w_{k}^{2}\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{v}^{2}} \right)}} + {\lambda\left( {{\sum\limits_{k - 1}^{k}{w_{k}c_{k}}} - 1} \right)}}} & (9)\end{matrix}$The partial derivatives of the Lagrangian are as follows

$\begin{matrix}{{\frac{\partial L}{\partial w_{k}}\left( {\left\{ w_{k} \right\}_{{k = 1},}^{k}\lambda} \right)} = {{2{w_{k}\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}} + {\lambda\; C_{k}}}} & (10) \\{and} & \; \\{{\frac{\partial L}{\partial w_{k}}\left( {\left\{ w_{k} \right\}_{{k = 1},}^{k}\lambda} \right)} = {{\sum\limits_{k = 1}^{k}{w_{k}C_{k}}} - 1}} & (11)\end{matrix}$Setting

$\frac{\partial L}{\partial w_{k}}$({w_(k)}_(k=1,) ^(k)λ) to zero yields the solution

$\begin{matrix}{{\hat{w}}_{k} = {{- \lambda}\frac{C_{k}}{2\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}} & (12)\end{matrix}$Substituting the solution of (12) into (11) and setting to zero yields

$\begin{matrix}{\hat{\lambda} = {- \frac{1}{\sum\limits_{k}^{K}{= \frac{C_{k}^{2}}{2\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}}}} & (13)\end{matrix}$Substituting (13) into (12) yields the following optimal weights:

$\begin{matrix}{{\hat{w}}_{k} = \frac{C_{k}}{\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right){\sum\limits_{k^{\prime} = 1}^{K}\frac{C_{k^{\prime}}^{2}}{2\left( {{C_{k^{\prime}}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}}} & (14)\end{matrix}$Next, we will analyze two cases of special interest.Detector Noise ModelUnder the assumption that the detector noise dominates the absorbancenoise, it holds that σ_(ν) ² can be set to zero with minimal error. Inthis case (14) becomes

$\begin{matrix}{{\hat{w}}_{k} - \frac{C_{k}}{\sum\limits_{k^{\prime} = 1}^{K}C_{k^{\prime}}^{2}}} & (15)\end{matrix}$From (15), it holds that, in presence of detector noise only, theoptimal strategy is to weight more heavily traces that have larger C_(k)². Note that the term Σ_(k′=1) ^(K)C_(k) ², is just a constant that canbe neglected if a multiplicative baseline is fitted.

Indeed, under this choice of the weights, the overall variance of theaveraged trace Ī is

$\frac{\sigma_{v}^{2}}{\sum\limits_{k = 1}^{K}C_{k}^{2}}$Observe that a simple unweighted sum will end up with a variance of

$\frac{K}{\left( {\sum\limits_{k = 1}^{K}C_{k}} \right)^{2}}\sigma_{e}^{2}$which is uniformly larger than

$\frac{\sigma_{v}^{2}}{\left( {\sum\limits_{k = 1}^{K}C_{k}} \right)^{2}}.$Fluctuation Noise ModelUnder the assumption that the detector noise can be neglected, i.e.σ_(ν) ²=0, it holds that (16).

$\begin{matrix}{= \frac{1}{{KC}_{k}}} & (16)\end{matrix}$From (16), it holds that, in absence of detector noise the optimalstrategy is to down-weight traces that have larger C_(k).

Under this choice of the weights, the overall variance of the averagedtrace Ī is

$\frac{\sigma_{v}^{2}}{K}.$Observe that an unweighted sum produces a variance of

$\frac{\sum_{k = 1}^{K}C_{k}^{2}}{\left( {\sum_{k = 1}^{K}C_{k}} \right)^{2}}\sigma_{v}^{2}$which is uniformly larger than

$\frac{\sigma_{v}^{2}}{K}.$

It is thus shown that the optimal averaging strategy depends on the typeof noise expected in the analyzer system. If the detector noise isnegligible, the received signal should be normalized by amplitude. Onthe other hand, if the detector noise dominates, the received signalshould be multiplied by the amplitude.

If both effects contribute significantly, the optimal weights are givenby (14).

Weighted Average Based, on Similarity to Adjacent Signals in the TimeSeries

In addition to assigning a weight to each TDLAS trace based uponsingle-trace characteristics (e.g. based upon the single-trace amplitudeCk), the weighting factor can be assigned based upon the similarity ordifferences of each signal to the prior and following signals in thecontinuous time series. Fast changes in signal amplitude are caused byprocess changes (e.g. a large dust particle or turbulent eddy) and canheavily distort a single ramp signal. Particularly if the processchanges occur on a time scale faster than the repetition rate of theTDLAS signal, a simple amplitude weighting can fail to rejectlow-quality signals. Several weighting schemes based upon prior andfollowing traces can be envisioned. In particular, a heuristic approachthat has been found to be effective is to calculate the deviationbetween consecutive ramps. In this approach, the difference between twotraces is calculated, the difference vector is squared, and the squareddifferences are summed. The weight is then the inverse of this quantity.This quantity can be computed based upon the difference with tracesadjacent in time either before, after, or including both before andafter the current TDLAS signal. This scheme can be combined with theweighted average based upon signal amplitude outlined in Equations (14),(15) and (16). Here, the total weight can be evaluated by e.g.multiplication of the weights based on signal amplitude and on signalsimilarity.

A weighted averaging scheme that uses the weights prescribed byEquations (14), (15) and (16) can be implemented as part of the signalprocessing algorithm for a tunable diode laser absorption spectroscopy(TDLAS) sensor. This weighted averaging scheme reduces sensor noise andimproves sensor performance in many cases, including dusty or turbulentenvironments, as compared to a simple unweighted average as used byother absorption sensors. In order to implement this strategy, threesteps must be undertaken: (1) determine the amplitude fluctuation ofeach trace C_(k); (2) calculate a weighting factor for each trace usingthe appropriate choice of the above strategies; and (3) calculate aweighted average of the traces using the prescribed weights. Theamplitude fluctuation, C_(k), for each trace can be determined viamultiple strategies including:

-   -   Peak-to-peak signal amplitude    -   Signal amplitude at a specific point or during a specific        duration of time in the TDLAS trace    -   Signal amplitude at a specific point or during a specific        duration of time in the TDLAS trace, minus any background signal        (determined during a time when the laser is disabled)    -   Slope of the ramp portion of the TDLAS signal        Using the determined amplitude of each trace C_(k). from one of        the above strategies, the user can determine if Equation        (14), (15) or (16) is appropriate to calculate the optimal        weighting coefficients based upon the known noise sources of the        instrument, and apply the weighting coefficient as part of a        weighted average calculation before passing the data to the        signal processing module.        Summary of Choice of Optimal Weighting Coefficients        Modeling the optimal weighting coefficient for each laser scan        by choosing the appropriate one of:    -   For the case where the noise is dominated by electronic sources        independent of the measured volume, the optimal weighting        coefficients are given by the equation:

$- \frac{C_{k}}{\sum_{k = 1}^{K}C_{k}^{2}}$

-   -   For the case where the noise is dominated by changes in the        measurement volume, the optimal weighting coefficients are given        by the equation:

$= \frac{1}{{KC}_{k}}$

-   -   For the general case, including both sub-cases above, the        optimal weighting coefficients are given by the equation:

$= \frac{C_{k}}{\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right){\sum_{k^{\prime} = 1}^{K}\frac{C_{k^{\prime}}^{2}}{2\left( {{C_{k^{\prime}}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}}$The general case is normally used only where neither particular noisesource dominates.

In each of the above equations,

is the optimal weighting coefficient, C_(K) is the amplitude of eachlaser scan k, σ_(e) ² is the variance of the electronic noise source,and σ_(ν) ² is the variance of the noise source within the measurementvolume.

Which types of noise are dominant in a measurement, and consequentlywhich weighting scheme is optimal, is typically known a priori basedupon the application. For an application with a laser having low poweror one carried out in a relatively clean environment (not a lot of dust,transmission losses, etc.) we would expect the system noise to bedominated by electronic noise. This is actually a very small number ofthe potential applications. For most applications, the dust andtransmission noise will dominate, and we would use the fluctuation noisemodel.

Using this spectroscopy system, one can improve operation andmeasurements results of an embodiment such as that of FIG. 5. FIG. 7illustrates a process in accord with the present invention ofpre-processing the raw data from a TDLAS system for effecting noisereduction in the data. Process 300 includes receiving data points (310),determining types or noise likely present in data points (320),calculating parameters for calibration (330, 340 and 350), andcalibrating the measurement system (360). Data points in a raw, sampleor measurement format come from the system at module 310. The datapoints may come in more than one format, allowing for determination ofwhat processing has occurred. A determination is made at step 320 as towhether the type of noise in the data points primarily representsdetector noise, fluctuation noise, or a mix of the two types of noise.

Depending on the results of this determination 320, differentcalculations of weighting coefficients for the signal averaging aremade. If detector noise dominates, then high magnitude data points areover-weighted in step 330 as with equation (15). If fluctuation noisedominates, then low magnitude data points are over-weighted in step 340as with equation (16). If neither type of noise dominates, a moregeneral approach to calculating coefficients is used in step 350 as withequation (14). The averaging calculations are then made at step 360based on the weighting coefficients from one of the steps 330, 340 and350. Note that the determination in step 320 may be made based on reviewof data points provided to the process. 300, or it may be made based onuser review and input, for example.

FIG. 8 illustrates an embodiment of a process of operating a TDLASsystem using the measurement process of FIG. 7. Process 400 includes thebasic modules or steps of process 200 of FIG. 6, along with additionaloperations. Process 400 further includes pre-processing calibration ornoise reduction request 485 for the raw data and pre-processingcalibration results 490.

Process 400 operates the TDLAS system as with process 200, processingmodule 210 through 260, thereby supplying samples, illuminating thesamples and collecting data. However, process 400 then requestscalibration or raw data pre-processing at step 485, invoking a weightedaveraging process such as that of process 300 of FIG. 7. At step 490,calibration results from that averaging are received by the process 400,such as through a signal that the system has been calibrated withadjusted parameters, for example, or a signal to enter adjustedparameters in the system based on calibration. At step 270, thepre-processed measurement data is further processed or analyzed. Thismay involve only processing new (i.e., changed) data as thespectroscopic system continuously operates. However, it may also involvere-processing and representing older raw or pre-processed data (e.g.,using a moving averages scheme), thereby re-evaluating or constantlyupdating such data and potentially providing a more correct orup-to-date result than was originally achieved with earlier measurementsand analyzing such updated data. The process also records and presentsthe processed data sample points at module or step 280, with or withoutadditional analysis results, for example, visually on a computerdisplay.

The invention claimed is:
 1. A spectroscopic instrument comprising: atleast one detector to collect a series of raw intensity measurement datapoints; and a processor to: calculate, based on a determination ofwhether detector noise or fluctuation noise dominates in the series ofraw intensity measurement data points, a set of weighting coefficientsfor the raw intensity measurement data points, wherein greater weight isgiven to larger intensity raw measurement data points over smaller rawintensity measurement data points when detector noise dominates, andwherein greater weight is given to smaller intensity raw measurementdata points over larger intensity raw measurement data points whenfluctuation noise dominates; and perform a weighted averaging upon theraw intensity measurement data points using the calculated set ofweighting coefficients to output, from the spectroscopic instrument, aset of pre-processed measurement data that is characterized by reducednoise contributions.
 2. The spectroscopic instrument of claim 1, whereinthe at least one detector is to successively receive new raw intensitymeasurement data points, and the processor is further to perform updatedweighted averaging calculations with the new raw intensity measurementdata points to output updated pre-processed measurement data.
 3. Thespectroscopic instrument of claim 1, wherein the processor is to applyweights $= \frac{C_{k}}{\sum_{k^{\prime} = 1}^{K}C_{k^{\prime}}^{2}}$when detector noise dominates.
 4. The spectroscopic instrument of claim1, wherein the processor is to apply weights $= \frac{1}{{KC}_{K}}$ whenfluctuation noise dominates.
 5. The spectroscopic instrument of claim 1,wherein the processor is to apply weights$= \frac{C_{k}}{\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right){\sum_{k^{\prime} = 1}^{K}\frac{C_{k^{\prime}}}{2\left( {{C_{k^{\prime}}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}}$when neither detector noise nor fluctuation noise dominates.
 6. Thespectroscopic instrument of claim 1, wherein the processor is a digitalsignal processor, and wherein the spectroscopic instrument furthercomprises: a sample chamber to receive a sample; a tunable laser diodeto illuminate the sample chamber; and a laser control module connectedto the tunable laser diode to control an output wavelength of thetunable laser diode.
 7. A tunable diode laser absorption spectroscopy(TDLAS) instrument comprising: a tunable laser diode to perform a seriesof laser scans through a sample gas; at least one detector to measure atransmission amplitude for each laser scan output as a series of rawintensity measurement data points; and a processor to: model noise inthe raw intensity measurement data points to select weightingcoefficients for each laser scan, wherein the selected weightingcoefficients are given as$= \frac{C_{k}}{\sum_{k^{\prime} = 1}^{K}C_{k^{\prime}}^{2}}$ in thecase where the noise is dominated by electronic sources independent ofmeasured volume, wherein the selected weighting coefficients are givenas $= \frac{1}{{KC}_{K}}$ in the case where the noise is dominated bychanges in measurement volume, and wherein the selected weightingcoefficients are given as$= \frac{C_{k}}{\left( {{C_{k}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right){\sum_{k^{\prime} = 1}^{K}\frac{C_{k^{\prime}}}{2\left( {{C_{k^{\prime}}^{2}\sigma_{v}^{2}} + \sigma_{e}^{2}} \right)}}}$in the general case where neither electronic noise nor measurementvolume fluctuations dominate the noise, wherein, in each of the aboveequations,

is the selected weighting coefficient, C_(K) is the measuredtransmission amplitude of each laser scan k, σ_(e) ² is variance ofelectronic noise, and σ_(ν) ² is variance of noise within themeasurement volume; perform a weighted average calculation using theselected weighting coefficients upon the raw intensity measurement datapoints to output a resulting spectrum from the TDLAS instrument with areduced noise contribution in the measurement data points; and analyzethe resulting spectrum output from the TDLAS instrument to produce adesired measured quantity for the sample gas.
 8. The TDLAS instrument ofclaim 7, wherein the processor is to reduce noise caused by one or moreelectrical sources.
 9. The TDLAS instrument of claim 7, wherein theprocessor is to reduce noise caused by variations in a process flow. 10.The TDLAS instrument of claim 9, wherein the processor is to reducenoise caused by attenuation due to dust or particulate.
 11. The TDLASinstrument of claim 9, wherein the processor is to reduce noise causedby density gradients in the process flow.
 12. The TDLAS instrument ofclaim 9, wherein the processor is to reduce noise caused by time-varyingtransmission losses.
 13. The TDLAS instrument of claim 7, wherein theprocessor is to determine a peak-to-peak signal amplitude for each laserscan.
 14. The TDLAS instrument of claim 7, wherein the processor is todetermine a signal amplitude for each laser scan at a predefinedmeasurement point or during a predefined duration of time during a TDLAStrace.
 15. The TDLAS instrument of claim 14, wherein the processor is todetermine a signal amplitude with a background signal subtracted foreach laser scan.
 16. The TDLAS instrument of claim 7, wherein theprocessor is further to determine a slope of a ramp portion of a TDLAStrace.
 17. The TDLAS instrument of claim 7, wherein the processor isfurther to adjust selected weighting coefficients for each laser scanbased upon similarity to one or more time-adjacent measurement traces,the adjustment comprising: calculating differences between a currentmeasurement trace and a time-adjacent measurement trace; squaring thedifferences; calculating an additional weighting factor by summing thesquares and taking the inverse of the sum; and multiplying theadditional weighting factor by the selected weighting coefficients toobtain adjusted weighting coefficients to be applied in the weightedaverage calculation.
 18. One or more non-transitory machine-readablemedia comprising a plurality of instructions stored thereon that, whenexecuted, cause a spectroscopic instrument to: collect a series of rawintensity measurement data points; calculate, based on a determinationof whether detector noise or fluctuation noise dominates in the seriesof raw intensity measurement data points, a set of weightingcoefficients for the raw intensity measurement data points, whereingreater weight is given to larger intensity raw measurement data pointsover smaller raw intensity measurement data points when detector noisedominates, and wherein greater weight is given to smaller intensity rawmeasurement data points over larger intensity raw measurement datapoints when fluctuation noise dominates; and perform a weightedaveraging upon the raw intensity measurement data points using thecalculated set of weighting coefficients to output from thespectroscopic instrument a set of pre-processed measurement data that ischaracterized by reduced noise contributions.
 19. The one or morenon-transitory machine-readable media of claim 18, wherein theinstructions further cause the spectroscopic instrument to: successivelyreceive new raw intensity measurement data points; and perform updatedweighted averaging calculations with the new raw intensity measurementdata points to output updated pre-processed measurement data.
 20. Theone or more non-transitory machine-readable media of claim 18, whereinthe instructions further cause the spectroscopic instrument to applyweights $= \frac{C_{k}}{\sum_{k^{\prime} = 1}^{K}C_{k^{\prime}}^{2}}$when detector noise dominates and to apply weights$= \frac{1}{{KC}_{K}}$ when fluctuation noise dominates.